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TITLE I PRESCHOOL PROGRAM IN THE 
WAKE COUNTY PUBLIC SCHOOL SYSTEM (WCPSS): 

SHORT- AND LONG-TERM OUTCOMES 

Contact: Nancy Baenen 


ABSTRACT 

The longitudinal study of the 2005-06 preschool in WCPSS found short-term gains during the 
preschool year, but limited impact by kindergarten and no average impact by the end of 3'^'^ grade on 
achievement, retention rates, special education placements, or attendance. Small sample sizes limit 
conclusions that can be drawn; a larger study with a stronger research design is recommended for 
next year. 

The larger 2010-11 preschool program provided strong evidence of short-term gains on achievement 
after the preschool experience. Teachers and parents were generally satisfied with the program. 
Teachers suggested higher student attendance and greater parent involvement would be helpful. 
Teachers used Creative Curriculum materials with varying frequency; they also viewed some 
components of the curriculum as more effective than others. 


BACKGROUND 


High quality preschool programs have been found to have a long-term positive impaet on 
student outcomes (Barnett, 1998; Li, Farkas, Dunean, Burehinal, Vandell, & Ruzek, 2011; 
Soriano, Duenas, & Leblane, 2006). Barnett (1998) found model early childhood programs 
produced slightly larger effeets on student outeomes than public school programs. 


Cost-benefit analyses of eight preschool programs across the 
eountry found high quality presehools provided benefits of $4 
to $7 for every $1 invested in terms of improved school 
readiness, better sehool performance, higher graduation rates, 
lower rates of crime and delinqueney, deereased dependence on 
welfare, and higher ineomes in adulthood (University of 
Pittsburgh, 2003). These benefits are seen only for low-income 
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students and only for high quality programs (described as having educated and experienced staff, 
higher payrolls, favorable staff to student ratios, and low teacher turnover). 

The Title I preschool program in WCPSS was developed to help academically needy preschool 
students be more successful upon entering kindergarten and beyond. Specific goals of the 
program are as follows. 

• Short-term goals of the program are that WCPSS entering kindergarteners who attend a Title 
I preschool will need less support than similar students, as evidenced by lower special 
education placements and retention rates. 

• The long-term goal of the program is that children who attend preschool will be more 
successful in grade than a comparison group of similar students and will be more likely to 
be on grade level as measured by the End of Grade 3'^'* grade test. 

The preschool program is designed to present a quality preschool experience including: a 
developmentally appropriate environment, parent involvement, training for teachers, certified 
teachers and teacher assistants, and curriculum that addresses the five developmental domains. 
The five developmental domains include: social, emotional, cognitive, physical, and language. 
WCPSS has had a preschool program within Title I for many years, but there have been a small 
number of classrooms until recently. In 2005-06, there were eight Title I preschool classrooms. 
During the 2009-10 school year, the Title I preschool program underwent a sizable expansion to 
39 classrooms with the use of additional Title I funds and American Recovery and Reinvestment 
Act (AREA) funds. Twelve classrooms were funded through Title I and the rest through AREA 
temporary funds. 

Table 1 below summarizes the overall budget for 2010-11 through these two sources. Overall, 
$4,577,757.51 was allocated to the preschool program in 2010-11. Some of the costs were one 
time start-up costs (close to $1,000,000 in 2010-11) which will not be a continuing cost. The 
continuing cost of providing a preschool classroom is about $95,236 per year for one teacher, 
one teacher assistant, supplies, and miscellaneous expenses. Each classroom typically serves 15- 
18 children. If we use 16.5 students and $95,236 as a typical cost per year, the approximate cost 
per student is $5,771 .88. This covers a full-day program. 
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Table 1 

Preschool Allotments 2010-11 


Fund 

Purpose 

Allocation 

Title 1 50 

Salaries and Benefits 

$1,292,642.77 

50 

Workshop Expenses/Allowable Travel 

$10,000.00 

50 

Printing and Binding Fees 

$10,000.00 

50 

Travel Reimbursement 

$12,000.00 

50 

Field Trips 

$14,000.00 

50 

Supplies and Materials 

$93,000.00 

50 

Supplies and Materials 

$35,855.00 

50 

Furniture and Equipment 

$65,000.00 

50 

Computer Equipment 

$76,000.00 

$1,608,497.77 

ARRA 141 

Salaries and Benefits 

$2,367,122.46 

141 

Printing and Binding Fees 

$0.00 

141 

Travel Reimbursement 

$2,000.00 

141 

Field Trips 

$0.00 

141 

Supplies and Materials 

$160,137.28 

141 

Furniture and Equipment 

$140,000.00 

141 

Computer Equipment 

$300,000.00 



'$2,969,259.74 


Data Source: Oracle printouts and Title I Director 


The Title I preschool sites use a curriculum that combines aspects from a number of curricula 
including: Opening the World of Learning (OWLS), The Creative Curriculum for Early 
Childhood, Second Step, and City Steps. During the 2010-11 school year, 36 of the classrooms 
had 18 children enrolled, two had 16 children enrolled, and one had 6 children (this class was 
combined with a hearing impaired preschool classroom). All of the 39 classrooms have at least 
one teacher’s assistant in addition to the main teacher. During the 2005-06 school year each of 
the eight classrooms had approximately 16 children and every classroom had a teacher and 
teacher’s assistant. 


EVALUATION METHODS 

The evaluation addressed both the long-term impact of preschool in WCPSS and the short-term 
impact of the expanded preschool program. 

LONGITUDINAL STUDY 

The purpose of the Title I Preschool Evaluation was to examine the effectiveness of the 
preschool program to students’ long-term success in school. The evaluation examined whether 
students who attended the Title I preschool program in the 2005-06 school year made academic 
progress during preschool, and then compared preschool students’ success in kindergarten and 
3'^‘* grade to a comparison group of students who were waitlisted for the program based on 
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screening measures. The 2005-06 year was ehosen because prior years included only five 
elassrooms and would not provide an adequate sample size. Students in 2005-06 would also 
have reaehed 3"^'* grade with normal progression by the 2009-10 school year. 

As is typieal in a longitudinal evaluation, some students were lost to the sample over time. Data 
were obtained for 62 of the 137 students who attended the presehool program during the 2005-06 
school year and who were still in WCPSS at the end of their 3'^‘* grade year (the 2009-10 school 
year). In addition, data were obtained for a eomparison group of 3 1 ehildren who were waitlisted 
for the program and were still in WCPSS at the end of their 3“ grade year. Children in the 
waitlist group were matched to those in the Title I preschool group on the edueational neediness 
seore. This neediness seore was derived based on results of sereening using the Briganee early 
ehildhood measure, whieh eovers language, motor, self-help, soeial-emotional, and eognitive 
skills. 

Three main measures were used to assess progress during the preschool year. The Creative 
Currieulum Developmental Continuum Assessment, Phonologieal Awareness Literaey Sereening 
(PALS), and The Briganee Preschool Screen were all given at the beginning and end of the 
preschool year. However, pre and post data for each measure were not available for all eight 
preschool sites. Counts were reviewed, and the deeision was made to use the five sehools with 
the most eomplete data. These five sites included; Brentwood Elementary, Creech Road 
Elementary, Projeet Enlightenment, Smith Elementary, and Vanee Elementary. Table 2 
summarizes the measures and eounts of seores and data available for eaeh group at grades pre-K 
and K. The Briganee provided initial information to establish eomparable groups and the 
Kindergarten Initial Assessment (KIA) provided a short-term impaet measure for the WCPSS 
presehool. The KIA is given to all ineoming kindergarten students within the first few weeks of 
sehool and eovers literaey, soeial development, and several other areas. Other preschool 
measures were utilized to measure growth for presehool partieipants on measures most elosely 
tied to the presehool eurriculum. Data were available for End-of-Grade (EOG), attendanee, 
speeial edueation plaeement, and retention rates at 3’^‘* grade for both groups. 


Table 2 

Breakdown of Sample Sizes Available for Instruments, 2005-06 


Grade 

Preschool Measures 

K 

2005-06 

Briganee 

(pre) 

Preschool 

Attendance 

PALS 

Creative 

Curriculum 

Briganee 

(post) 

KIA 

(post) 

Title I 
Preschool 

62 

62 

37 

62 

61 

55 

Waitlist 

31 

— 

— 

— 

— 

30 


Data Source: Data files for each instrument were provided by Title I preschool staff 
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The Creative Currieulum Developmental Continuum Assessment was eondueted in the fall and 
the spring and data were available for all 62 of the presehool students in the sample. This 
measure is designed to assess four main areas of development: Soeial/Emotional, Physieal, 
Cognitive, and Language. This measure uses a eontinuum that is based on the expeeted 
developmental steps during the presehool years to assess eurrieulum objeetives. This eontinuum 
allows for assessment of progress over time on eaeh objeetive. 

The Phonologieal Awareness Literaey Sereening (PALS) was eondueted in the fall and the 
spring and data were available for 37 of the presehool students in the sample. Tasks on the 
PALS are designed to be developmentally appropriate for 4 year olds and assess four main areas: 
Phonologieal Awareness, Alphabet Knowledge, Print and Word Awareness, and Name Writing. 
This measure has demonstrated high levels of reliability - split half reliability, r = 1 .00; Internal 
eonsisteney, alpha = 1.00; inter-rater reliability = .99. Additionally, eontent, eonstruet and 
eriterion-related validity have all been established. 

The Briganee Presehool Sereen was eondueted in the fall and the spring and data were available 
for all 62 of the presehool students in the 2005-06 sample expressed as standard seores. The 
assessments in this seetion allow sereening personnel to assess the basie skills of four-year-old 
presehool ehildren. Additionally, only the 4 year old version was used, regardless of whether 
ehildren were 4 at the time of the assessment or not. This measure has demonstrated high levels 
of reliability (internal eonsisteney from .84 to .89, test-retest with r from .84 to .99, inter-rater 
reliability from .90 to .99). The Briganee Presehool Sereen has also demonstrated aeeeptable 
levels of eontent, eonstruet, eoneurrent, predietive, and diseriminant validity. 

In order to assess the short-term impaet of presehool, ehildren’ s seores on the KIA were used. 
The KIA is given eountywide to ineoming kindergarteners eaeh fall in order to assess students’ 
skill level. A subset of the KIA items that are most relevant to the eurrieulum used in the Title I 
presehools, will be used in this evaluation. The items seleeted for this evaluation ineluded eight 
early literaey items (Print Coneepts, Retelling, Letter ID, Letter Sound, Word ID, Coneept of 
Spoken Word, Rhyme Reeognition, Rhyme Produetion), four Math items (Rote Counting, One- 
to-One, Geometry, Patterns), five language items (Takes Care of Personal Needs, Reeognizes 
and Responds to Name, Interaets with Adults and Peers, Responds to Adult Direetion, Lollows 
Sehool Routines), and four Personal/Soeial items (Starts Conversation, Communieates Personal 
Experienee, Asks Questions, Talks While Pretending). In addition to the KIA, ehildren’s 
attendanee reeords and retention reeords were obtained for the 2006-07 kindergarten year. 

When examining the long-term goal of the program, the grade LOG seores for math and 
reading were used to assess the presehool and waitlisted students’ status in 3’^‘* grade. The 
ehildren who attended the Title I Presehool Program in 2005-06 should have been in 3"^*^ grade 
during the 2009-10 sehool year. In addition to LOG seores, students’ attendanee reeords, Speeial 
Edueation plaeement, and retention reeords were used to examine the long-term impaet of the 
Title I Presehool Program. 
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2010-11 PRESCHOOL PROGRAM 

During the 2010-11 school year, additional methods were used to assess the state of the current 
preschool program and the progress of the current students. A survey of the 2010-1 1 Title 1 
Preschool Teachers was conducted in April, 2011. This survey was designed to acquire 
information from teachers about their perceptions of the effectiveness of program components, 
implementation of the program, and satisfaction with program training, materials, and support. 
The survey was sent via Survey Gizmo by program staff to all 39 preschool teachers (one teacher 
at each preschool site) via an online survey tool, and teachers were able to complete the survey 
online. Twenty-nine teachers completed the survey before the deadline, which equates to a 74% 
response rate. 

Additionally, a survey was given to parents of children who participated in the 2010-11 Title I 
Preschool Program. The survey was conducted at the end of the school year in May 2011 by the 
preschool staff. Parents were able to complete this survey either online or by completing a paper 
copy during a family-school partnership day at their child’s preschool site. Additionally, the 
survey was translated into Spanish so that Spanish-speaking parents were also able to complete 
it. The survey was designed to acquire information from parents about the effectiveness of the 
program, their satisfaction with the program, and their level of participation in program events. 
There was approximately a 37% response rate for the parent survey. Because respondents were 
not asked to supply their school name, it was not possible to determine if responses reflected all 
sites within the program. Spanish speaking families accounted for 21% of the survey responses. 
While this return rate is not unusually low for a parent survey distributed online or through a 
mail out process, responses will be shared on a limited basis because we cannot assess the 
representativeness of the returns. 

Finally, the Brigance Preschool Screen was used to assess progress for 2010-11 preschool 
students. Standard scores, percentiles, and age equivalents were available for this group of 
students. Additionally, for this group of students, the four-year old version was only used when 
children were still four years old at the time of the assessment; if the child had turned five before 
the assessment then the five-year old version was used. Due to the fact that different versions 
may have been used for pre and post assessment (because of child’s age) the percentile scores 
and age equivalents will be used to validly assess student growth from pre- to posttest. 

LIMITATIONS 

A key limitation of the longitudinal part of this study is the small sample size. While we initially 
had 137 students in the preschool program in 2005-06, we were unfortunately only able to use 
data for five of the eight campuses because three campuses had incomplete data. Students were 
also lost to attrition over time, so by the end of 3'^‘* grade, 62 students remained in the sample 
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with 30 matching comparison students from the waiting list. Therefore, this study must be 
eonsidered more exploratory than eonelusive. 

Another limitation of the eomparison group is the parents’ designation of whether their ehild 
reeeived presehool serviees was the only measure available for this important variable. WCPSS 
does not have reeords of ehildren’s partieipation in presehools aeross the eounty. Parents 
sometimes eonsider day eare serviee to be the same as presehool, but the quality is often quite 
different. 

Finally, data are not available on the quality of the implementation of the program in 2005-06. 
This ean make it diffieult to distinguish whether implementation levels are related to the 
findings. Data for 2010-11 is limited to survey self-report data; observation data would be 
stronger evidenee of the quality of implementation. 

LONGITUDINAL STUDY FINDINGS 

Did Preschool Participants Show Growth After the Preschool Experience? 

Available assessment measures suggest preschool students made considerable growth during 
preschool. However, it was not possible to compare this growth to a comparable group or to 
national norms. 

Student assessment results before and after the presehool year were initially examined to assess 
student growth. The average number of days absent for the sample of presehool students from 
the 2005-06 sehool year was 16 days (out of 180). Although there is no speeifie eomparison for 
this number, generally having over 10 absenees ean be eonsidered problematie. A series of pre 
and post assessments were eompleted during the 2005-06 sehool year on the Title I presehool 
group. Waitlisted students had only some presereening measures and no tests at the end of the 
presehool period, so eomparisons were not possible. Measures that offer developmental norms 
provided the best measure of presehool sueeess, beeause they provided a standard for whether 
students grew more than would normally be expeeted. 

The first measure used to assess progress over the eourse of the presehool year was the Briganee 
Presehool Sereen. A paired samples t-test showed signifieant growth between the pretest seores 
(Mean = 35.11) and posttest seores (Mean = 75.69) (t= -19.629, p=. 000, n= 61). 

The seeond measure given was the PALS. Seven subtests were ineluded on the pre- and posttest: 
Name Writing, Upper Case Reeognition, Lower Case Reeognition, Letter Sounds, Beginning 
Sounds Awareness, Print/Word Awareness, and Rhyme Awareness. Paired sample t-tests 
revealed that posttest seores on all subtests were signifieantly higher than pretest seores (see 
Table 3). Additionally, at least 65% of students seored at or above the spring developmental 
range seores on all sub tests (a fall developmental range was not available for eomparison). 
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Name Writing and Beginning Sound Awareness had the strongest results, with over 86% of 
seores above the spring developmental range. 


Table 3 

PALS Results - Fall and Spring 2005-06 


PALS Subtest 

T score 

Significance 

Fall Mean 
Score 

Spring 
Mean Score 

% at or above 
Spring 

Developmental 

Range 

Name Writing (n=36) 

-6.454 

p =.000 

3.61 

6.31 

89.2% 

Upper Case 
Recognition (n=37) 

-7.795 

p =.000 

5.70 

16.46 

75.7% 

Lower Case 
Recognition (n=37) 

-7.327 

p =.000 

3.19 

13.95 

70.3% 

Letter Sounds (n=37) 

-6.337 

p =.000 

1.59 

8.27 

64.9% 

Beginning Sound 
Awareness (n=37) 

-4.751 

p =.000 

4.97 

8.54 

86.5% 

PrintAVord Awareness 

(n=37) 

-5.897 

p =.000 

A.13, 

7.76 

75.7% 

Rhyme Awareness 

(n=37) 

-5.280 

p =.000 

3.30 

6.30 

64.9% 


The Creative Currieulum Assessment was the pre and post measure given during the 2005-06 
presehool year. Another series of paired samples t-tests revealed that the posttest seores were 
signifieantly higher than the pretest seores for all four areas (see Table 4). 


Table 4 

Creative Curriculum Results, 2005-06 


Creative Curriculum 
Subtest 

T score 

Significance 

Fall Mean 
Score 

Spring Mean 
Score 

Social/Emotional 

-12.215 

p =.000 

18.52 

32.34 

Physical 

-10.834 

p =.000 

12.76 

20.56 

Cognitive 

-15.369 

p =.000 

17.44 

36.11 

Language 

-14.720 

p =.000 

15.90 

30.95 


62 
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Thus, available measures show eonsiderable growth in students’ seores over time. However, 
national norms and/or a eomparison group are not available to put this growth in eontext. 

Did the 2005-06 Preschool Program Meet Its Short-Term Goals? (Kindergarten) 

KIA scores were significantly higher for preschool participants on two of 21 measures. 
Attendance, retention, and special education placements did not vary statistically. This provides 
little evidence of a major short-term impact of preschool. 

Title I Presehool students who attended the program during the 2005-06 sehool year were 
eompared to a group of students who were on the waitlist and not admitted into the Title I 
program. The two groups were found to have similar eharaeteristies. 

• Initial edueational neediness seores (a seore that was used as eriterion for admittanee to the 
program), for the two groups were not statistieally signifieantly different. Sinee the ehildren 
on the waitlist had similar seores to those who were admitted to the program, it is likely that 
they too would have been admitted had more openings been available. 

• The Title I presehool group and the waitlist group were demographieally similar in terms of 
pereentage of students on free or redueed-priee luneh (FRL) and limited English profieient 
(LEP) students (see Table 5). 

• Additionally, the two groups were similar in terms of the pereentage of students of different 
raees/ethnie groups (see Table 5). 


Table 5 

Characteristics of Preschool Participants and Waitlisted Students, 2005-06 



FRL 

LEP 

Black 

Hispanic 

White 

Other 

Title I Pre-K % 

76.2% 

28.6% 

42.9% 

36.5% 

20.6% 

0% 

Number 

48 

18 

27 

23 

13 

0 

Waitlist % 

77.4% 

29.0% 

54.8% 

22.6% 

16.1% 

6.5% 

Number 

24 

9 

17 

7 

5 

2 


Data Source: Title I data files (paper, converted to electronic Excel files). 


In addition, an item on the KIA was used to examine if the two groups were signifieantly 
different in presehool experienee. However, this variable may not be eompletely aeeurate for 
these two groups of students due to ineonsistent reporting of this variable. Sometimes parents 
eount day eare as presehool, and rosters from other presehools were not available. Presehool 
experienee is based on parent responses to a general question on the registration form. The 
presehool item is open to interpretation in the type, amount, and quality of pre-kindergarten or 
ehildeare that eonstitutes presehool experienee. Approximately 15% of the presehool sample 
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used in this study was reported as not having attended preschool, which we know was incorrect. 
In an effort to have this variable accurately reflect the preschool experience of the sample, the 
variable was corrected to show that all children in the preschool sample had attended preschool. 
When run with the variable as it was reported by parents, the chi square suggested that the two 
groups were not significantly different (Pearson Chi Square = 6.847, p=.144). Approximately 
60% of the waitlist group was reported to have had preschool experience. When run with the 
corrected variable the Chi square revealed that the groups were significantly different in their 
preschool experience (Pearson Chi Square = 24.173, p=.000). However, readers must be aware 
that some error likely exists in the waitlisted responses as well. 

Several measures from the kindergarten year were used to examine success in meeting the short- 
term goals of the program (KIA scores, special education placements, retention rates, and 
attendance rates. 

Upon entering kindergarten, children were given the KIA and therefore items from this measure 
are being used to compare the preschool and waitlist groups at the beginning of the kindergarten 
year. Of the selected KIA items, the preschool group was significantly different than the waitlist 
group on two items; Letter ID and Rhyme Production. Letter ID is considered a critical indicator 
of potential success in learning to read. For both of these items the preschool group 
outperformed the waitlist group. Additionally, the difference between the two groups on Rhyme 
Recognition was approaching significance, with the preschool group performing better than the 
waitlist group. For the remainder of selected KIA items, the two groups were not significantly 
different in their performance (see Tables 6 and 7). 


Table 6 

KIA Mathematics Items Chi Square Results, 2005-06 


KIA Items 

Chi Square 

Significance 

1. One-to-One 

1.227 

.747 

2. Patterns 

2.232 

.328 

3. Geometry 

3.481 

.481 


n = 55 for preschool and 30 for waitlist 
Data Source: E&R Data Files (Quickr) 
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Table 7 

KIA Literacy Items T Test Results, 2005-06 


KIA Items 

Preschool 

Mean 

Waitlist 

Mean 

T-test 

Significance 

1 . Print Concepts 

6.93 

5.63 

1.523 

.132 

2. Retelling 

1.75 

1.80 

-.289 

.773 

3. Letter ID 

28.82 

19.07 

2.548 

.013* 

4. Letter Sound 

6.35 

4.43 

.671 

.504 

5. Word ID 

.62 

.83 

-.356 

.722 

6. Concept of Spoken Word 

2.38 

2.27 

.273 

.786 

7. Rhyme Recognition 

3.24 

2.43 

1.839 

.070 

8. Rhyme Production 

1.64 

.43 

2.588 

.011* 

9. Starts Conversation 

2.78 

2.83 

-.313 

.755 

10. Communicates Personal Experience 

2.65 

2.57 

.510 

.611 

1 1 . Asking Questions 

2.44 

2.43 

.018 

.986 

12. Talks While Pretending 

2.82 

2.90 

-.541 

.590 

13. Rote Counting 

16.02 

14.63 

.704 

.484 

14. Recognizes and responds to name 

3.05 

3.07 

-.102 

.919 

15. Takes care of personal needs 

2.91 

3.03 

-1.192 

.237 

16. Interacts with adults and peers 

2.85 

2.93 

-.537 

.593 

17. Responds to adult direction 

2.87 

2.80 

.494 

.623 

18. Follows school routines 

2.80 

2.67 

.899 

.371 


n = 55 for preschool and 30 for waitlist 
* = Significant 

Data Source: E&R Data Files (Quickr) 

Short-term goals of the presehool program were that partieipants would need less support than 
similar students, as evideneed by lower speeial edueation plaeements and retention rates. 

At the beginning of kindergarten the two groups were not signifioantly different in the number of 
students with speeial edueation plaeements (Pearson Chi Square =.544, p=.762). At the end of 
the kindergarten year the two groups were also not signifioantly different for attendanoe or 
retention rates. 
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• Attendance; the Title I preschool group (Mean = 8.40) and the waitlist group (Mean = 6.90) 
did not have significantly different numbers of days absent during their kindergarten year 
(t= 1.092, p=.278). 

• Retention; The two groups were also not significantly different in the number of students 
that were retained (Pearson Chi Square = 4.053, p=.132). Few students were retained from 
either group. 

Has the Title I Preschool Program met its long-term outcomes/goals? 

EOG results do not support a long-term impact of the preschool program based on this small 
sample. Retention rates, attendance rates, and special education placements were also not 
different for those who attended preschool in WCPSS compared to similar students. 

The long-term goal of the program was that children who attended Title I preschools would be 
more likely to be on grade level in 3'^‘* grade than students not served. In order to determine the 
success in meeting this goal, EOG scores for the preschool and waitlist groups were compared 
(see Table 8). 

• Approximately 27% of the Title I preschool sample was on grade level in reading and 
approximately 54% were on grade level in mathematics. 

• However, when examining Reading EOG scores, the Title I preschool group was not 
significantly different from waitlist group (t= -.566, p=. 573). Similarly, Mathematics EOG 
scores for the Title I preschool group were not significantly different from scores for the 
waitlist group (t= -.676, /)=.501). 

• Both groups had a similar breakdown of scores for both reading and mathematics. While 
the scores are generally lower than for WCPSS as a whole, they are similar to the ERL and 
EEP subgroups. 
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Table 8 

Title I and Comparison Groups by EOG Level, 2005-06 


EOG 

Scores 

Level 

Title I 
Pre-K 

Waitlist 

WCPSS 

ERL 

LEP 

Reading 

4 

5.6% 

3.4% 

29.2% 

8.7% 

2.3% 


3 

21.8% 

24.1% 

43.2% 

38.5% 

21.7% 


2 

27.2% 

34.5% 

13.2% 

21.7% 

21.4% 


1 

45.4% 

37.9% 

14.4% 

31.1% 

54.5% 

Math 

4 

9.1% 

10.3% 

38.1% 

13.5% 

8.6% 


3 

45.4% 

44.9% 

47.1% 

55.7% 

47.6% 


2 

38.2% 

37.9% 

11.8% 

23.4% 

33.2% 


1 

7.3% 

6.9% 

3.0% 

7.3% 

10.6% 


n = 63 for preschool and 3 1 for waitlist 


In addition to EOG scores, attendance, retention, and speeial education placement data were used 
to examine whether presehool students fared better than those students in the waitlist group. Chi 
square results revealed no signifieant differenees in terms of the number of days absent during 
their 3’^‘* grade year, speeial edueation plaeements, or retentions at any point up to and ineluding 
their 3’^‘* grade year. Table 9 shows aetual values, whieh illustrate that differences in values can 
be relatively large (e.g., 7% in the case of kindergarten retention) without reaehing statistieal 
significanee) with these small samples. 


Table 9 

Title I Outcomes Comparison, 2005-06 


Outcomes 

Title I 
Pre-K 

Waitlist 

WCPSS 

ERL 

Black 

Hispanic 

Attendance 
(mean days absent) 

6.02 

6.67 

- 

- 

- 

- 

Special Ed. Placement 

17.5% 

11 

9.7% 

3 

- 

- 

- 

- 

K-3'^‘* Grade Retention 

14.8%* 

9 

16.7%* 

5 

- 

- 

- 

- 

Kindergarten Retention 

0% 

0 

7.4% 

2 

4.2% 

480 

0.5% 

55 

1.6% 

180 

1.3% 

152 

3’^'* Grade Retention 

.2% 

1 

9.7% 

3 

1.2% 

145 

0.2% 

26 

.5% 

60 

.4% 

42 


n= 62 for preschool and 3 1 for waitlist group 
* Percent of children retained in any grade up through 3"^^ grade 


13 




Title I Preschool Evaluation 


E&R Report No. 11.16 


2010-11 PRESCHOOL PROGRAM FINDINGS 

The 2010-11 preschool program provided data on implementation and short-term outcomes. 
Overall, 686 students were enrolled during the 2010-11 school year at 38 schools in 39 classes. 
The curriculum used was substantially the same as that used in 2005-06, but the number of 
classes was far greater. Teachers were hired from July 1, 2010 to June 30, 2011; training 
included topics related to curriculum and meeting the needs of individual students. Of the 
teachers who were hired 7.7 % were new to WCPSS, 12.8 % were new to preschool, and 
100% were certified to teach preschool. Each class had 15-18 students and was staffed with a 
teacher and at least one teacher assistant. 

Was the Preschool Program implemented as planned? 

Most teachers reported they had what they needed to implement the preschool program well. 
Actual observation data were not available. Reported frequency of use of the Creative 
Curriculum varied across teachers. 

Data on the observed implementation of the program in individual preschool classrooms were 
not available at the time of the report. However, some basic information on implementation of 
the program was available through teacher survey data. Creative Curriculum is the major 
curriculum resource for the program, and daily use is expected by central staff. Over two thirds 
(69%) of the teachers said they use the Creative Curriculum multiple times a day in their 
classrooms (31%) or daily (38%). However, 21% indicated only weekly use, 3% said they only 
use Creative Curriculum 2 to 3 times a week, and 7% said that they do not use it in their 
classrooms at all. 

Also, as shown in Table 10, teachers felt well supported in implementing the program. 

• In terms of support for preschool teachers, approximately 86% of teachers felt that they were 
supported by their Title 1 coordinating teacher and approximately 66% felt supported by the 
Title 1 central office team (see Table 10). 

• With regard to materials, 83% of teachers felt they had the materials they needed to meet the 
needs of their students. 

• Finally, 76% of teachers felt that the professional development that was provided was 
effective. 

Thus, these data indicate most teachers believed they had what they needed to implement the 
preschool program well. Close to one third reported using Creative Curriculum materials less 
often than expected. 
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Table 10 

Teacher Perceptions of Aspects of Program Implementation, 2010-11 


Items 

Strongly 

Agree 

Agree 

Neutral 

Disagree 

Strongly 

Disagree 

I have been supported by my Title 1 
Coordinating Teacher. 

34.5% 

51.7% 

13.8% 

0% 

0% 

I have been supported by the Title 1 
Central Office Team. 

20.7% 

44.8% 

24.1% 

6.9% 

3.4% 

I have the necessary materials to 
meet the needs of my students. 

27.6% 

55.2% 

0.0% 

13.8% 

3.4% 

The professional development 
provided by the Title 1 Pre-K 
Department was effective. 

27.6% 

48.3% 

17.2% 

6.9% 

0% 


n=29 

Data Source: Preschool Teacher Survey 


Did students show growth on the Brigance? 

Overall, preschool students did show growth on the Brigance, with gains beyond what would be 
expected normally after preschool service. 

During the 2010-11 school year, a number of measures were completed regarding the current 
preschool class. At the time of this report, the data available to measure progress over the course 
of the year were from the 2010-11 Brigance Screen. In order to compare total progress, the Total 
Quotient score was used because it allows for comparison to national norms and because it is the 
best type of score for an aggregate comparison (see Table 1 1). According to national norms, the 
pretest mean score of 77.50 fell in the Poor range; however, the posttest mean score of 98.66 fell 
in the Average range. Additionally, the Academic Age Equivalent for the posttest was 
significantly different from the pretest (t= -36.836, p=.000). The Language Age Equivalent for 
the poshest was also significantly different from the pretest (t= -15.616, p=.000). Thus, these 
data provide stronger evidence than the 2005-06 program that students showed growth beyond 
what would normally occur after participating in the preschool program. 


Table 11 

2010-11 Brigance Results, 2010-11 


Scores 

Pretest 

Mean 

Posttest 

Mean 

T test 

Significance 

Total Quotient 

77.50 

98.66 

-22.552 

.000 

Academic Age Equivalent 

(in months) 

41.57 

61.50 

-36.836 

.000 

Language Age Equivalent 

(in months) 

39.55 

55.37 

-15.616 

.000 


« = 201 
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What facets of the Preschool Program are viewed as most effective and least effective? 

Within the Creative Curriculum, teachers most often cited the Social/Emotional, Oral Language, 
and Cognitive aspects as most effective and the Social Studies and Science & Technology aspects 
as least effective. Teachers felt higher student attendance rates and parent participation rates 
would be helpful. Parents did tend to have conferences with the teachers, but participation in 
other ways was limited. Most parents who responded to the survey found the program helpful in 
preparing their children for kindergarten. 

The effectiveness of the curriculum can impact student outcomes. While the program uses parts 
of multiple curricula, the Creative Curriculum is one major part of the program curriculum, 
which has multiple components within it. Teachers were asked to comment on which 
components of the Creative Curriculum they found most effective and which they found to be 
least effective. As shown in Figure 1, teachers found the Social/Emotional, Oral Language, and 
Cognitive aspects of Creative Curriculum to be the most effective components, while they found 
Social Studies and Science & Technology to be the two least effective components. 

Mathematics and literacy components were in the mid-range of responses; it is interesting that 
about the same percentage of teachers selected mathematics as the most effective and least 
effective aspect of the program (33%). 


Figure 1 

Teacher Rated Effectiveness of Creative Curriculum Areas 
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Data Source: Preschool Teacher Survey 
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Student attendance can impact how effective a program is at increasing student achievement. 
According to teacher reports student attendance can be a problem; 28% of teachers reported that 
between 5 and 9 students in their classrooms had 10 or more absences. Additionally, 69% of 
teachers felt that attendance impacted student’s achievement. A few teachers also commented 
that there need to be stricter attendance policies in order to increase student attendance. 

Another important aspect of the program is the family-school partnership. As shown in Table 
12, two-thirds of teachers felt that the family-school partnership aspect of the program is a 
necessary part of the program, and that it does impact student achievement in a positive way. 
Teachers mentioned that parents who participated learned the importance of being involved in 
their child’s education and were able to continue to help their child learn at home. Additionally, 
the children of participating parents had fewer behavior problems in the classroom and did better 
academically. Of course, since participation is voluntary, parents who elect to participate may be 
more involved with their children’s development overall. 

Table 12 

Teacher Perceptions of Family Involvement, 2010-11 


Item 

Yes, 

Definitely 

Yes, 

Somewhat 

No 

Family-school Partnership Days are a necessary part 
of the Title 1 Pre-K program. 

65.5% 

31.0% 

3.5% 

Has family involvement impacted the achievement 
of students? 

69.0% 

24.1% 

6.9% 


n\ 29 

Data Source: Preschool Teacher Survey 


Despite the importance of this aspect of the program, the Director of the preschool program 
reported that participation in parent sessions varied by classroom considerably, with some 
teachers frequently reaching 100% attendance and others with 60% for their most highly 
attended session. Some teachers commented that lack of participation in these family-school 
partnership activities was an issue. A few of the teachers commented that some of the families 
were not consistent with their child’s attendance and did not consistently participate in family- 
school partnership activities. Teachers suggested that additional support from the Title I staff on 
these aspects of the program (e.g., through a letter sent to families from program staff) might be 
beneficial. Comments from some teachers suggest that family involvement is considered 
important by staff but participation rates by parents are lower than desired. 

The program goal for parent conferences is to have at least two per family and nearly all (90%) 
of teachers did indicate that they usually had one to three conferences with each family 
throughout the year. 
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While the parent survey return rate was low (37%), responses were provided by 255 parents. 
Responses to a few key questions merit discussion. Approximately 93% of the parents who 
responded to the survey were extremely satisfied with the preschool program. In addition to 
overall satisfaction, approximately 82% of responding parents felt that the Title I preschool 
program helped their child very much with their social skills. Approximately 67% of responding 
parents felt that the Title I preschool program helped their child very much with reading skills. 
Finally, 65% of responding parents felt that the Title I preschool program helped their child very 
much with their mathematics skills (see Table 13). 


Table 13 

Parent Perceptions of Program Effectiveness, 2010-11 


Items 

Very 

Much 

Quite A 
Bit 

Somewhat 

A Little 

Not At 
All 

How much did Title I Pre-K Program 
help your child with SOCIAL skills? 

81.9% 

16.1% 

1.6% 

0.4% 

0% 

How much did The Title I Pre-K 
Program help your child with 
READING skills? 

66.5% 

26.4% 

6.3% 

0.4% 

0.4% 

How much did the Title I Pre-K 
Program help your child with MATH 
skills? 

65.0% 

28.0% 

6.3% 

0% 

0% 


n=255 

Data Source: Preschool Parent Survey 


Parents were asked what that their favorite part of the program was; common open-ended 
responses included: 

• the teachers and their investment in the students, 

• the effort to involve parents and families in the program, 

• the chance for their children to learn social skills, 

• the academic progress that their child showed over the course of the year, and 

• the preparation of their child for kindergarten. 

How could the Title I Preschool Program be improved? 

The teacher survey asked teachers about their biggest challenges in implementing the preschool 
program. Common challenges included: meeting Early Childhood Environment Rating Scale 
(ECERS) standards, lack of support from administration at the school, difficulty in ordering 
supplies, lack of materials for the OWL curriculum, dealing with challenging behavior, and no 
duty-free lunch or planning time. 
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In addition, when asked about improvement on the teaeher survey, teachers mentioned the 
following as areas of improvement: 

• more consistency in school-central office expectations for classroom routines and 
procedures, 

• more support from coordinating teachers, meaningful and relevant professional development, 

• better/easier access to materials and resources for classroom, and 

• changing the parent workshops and family-school partnership days so that they are better 
attended. 

Based on responses from the parent survey, a large number of parents would not change anything 
about the Title I preschool program. Numerous parents commented that the program should be 
available to more children and be offered at more sites. Parents also commented that although 
the parent workshops and family-school partnership days are valuable to the program they 
should be more accessible to working parents, perhaps by holding one or more on a Saturday. 
Finally, parents commented that there could be better communication especially for Spanish 
speaking families. 

Is the WCPSS preschool program cost effective? 

Research studies such as that conducted by the University of Pittsburgh (2003) found high- 
quality preschools can be quite cost effective, returning $4-7 per dollar of investment over time. 
Some of these benefits can only be measured later in students’ educational careers and beyond 
(such as graduation rates, higher average incomes, and reduced delinquency and incarceration 
rates). Benefits which can be seen much early in students’ educational careers include school 
readiness for kindergarten and stronger achievement scores. 

Costs for preschool programs included in the University of Pittsburgh 2003 study ranged from 
about six to thirteen thousand dollars per year. Most were part-time programs, with some full- 
day programs. The WCPSS program cost was $5,772 if one-time start-up costs are not included; 
all were full-time programs. While the 2003 study notes that costs are calculated in differing 
ways across studies, the WCPSS program clearly appears to be at the low end of the cost range. 

In our WCPSS study, positive short-term benefits in terms of preschool learning were observed, 
with limited advantages in terms of kindergarten readiness. However, by 3'^'* grade, positive 
impacts on achievement, on the average, were not evident. Barnett (1998) found that 
achievement effects appear to fade out in studies without strong study designs and with 
considerable selective attrition, but do not fade out in experimental or quasi-experimental studies 
with few methodological constraints. Unfortunately, in this post-hoc study, it was impossible to 
avoid some methodological issues— it was not possible to create an experimental or quasi- 
experimental design, some of the comparison students were served in different preschool 
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programs, the impact of attrition on the remaining sample is unknown, and the small sample size 
reduces the statistical power to detect effects. 

Whether some of the long-term benefits will become evident over time is impossible to tell at 
this point. Based on evidence available at present, we cannot demonstrate long-term cost 
effectiveness based on higher achievement, lower retention rates, or lower special education 
referral rates as of 3“ grade. 


CONCLUSIONS 

The findings indicate that preschool participants do show academic growth beyond what would 
be normally seen during the preschool year and to a limited extent at the start of kindergarten. 
However, 3rd grade data on End of Grade, retention. Special Education placements, and 
attendance do not show that preschool participants are any more successful than similar students 
not served in the Title I preschool program. 

The teacher survey results indicate that teachers are fairly satisfied with working in the Title I 
preschool program; however, there is still room for improvement in areas such as student 
attendance, parent participation, and consistent implementation of the Creative Curriculum. 
Additionally, the parent survey results suggest that the majority of parents are satisfied with the 
program and feel that it helps their children in multiple areas. 

RECOMMENDATIONS 

WCPSS needs to know more about the benefits received from its investment of federal dollars in 
preschool. While national and experimental studies have found lasting effects of preschool 
experience on student achievement (Barnett, 1998; Soriano et al. 2006), our current data do not 
support that view for the WCPSS program. The WCPSS finding is not unique among studies 
without experimental designs and with small sample sizes. At a 2010-1 1 cost of $4.5 million, we 
need to have a better sense of whether our preschool efforts are cost beneficial as structured. If 
not, the preschool method currently used for low-income students might need to be enhanced or 
replaced with a stronger model. 

Examine aspects of model preschools and modify our program to match them more closely 
if appropriate. This small longitudinal study did not discern significant impact of our current 
program on students’ success through 3'^‘* grade. However, the small numbers of participants and 
comparison students is a very real limitation of this study, and we must consider it primarily 
exploratory. At this point, we recommend staff review model preschool programs with positive 
research results to determine practices, materials, or conditions we do not use and incorporate 
them as feasible. Title I central staff members are also encouraged to ask other districts or Title I 
staff at the state or federal level for ideas on how to promote parent involvement and student 
attendance. Preschool teachers within WCPSS might also be asked to brainstorm possibilities. 
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and perhaps try two different methods in an experimental way to eheck impaet. One idea 
revealed in the survey results was to ehange the timing of the parent sessions to promote higher 
attendanee. Teaehers who have high attendance at parent sessions may have some valuable ideas 
to share as well. 

Start a quality long-term study of preschool in WCPSS with next year’s cohort. Preschool 
programs can meet an important need, in helping to “level the playing field” for students from 
low income backgrounds whose families cannot afford quality preschool programs. The 
Curriculum Management Audit and Board Policy 5600 both cite the importance of evaluating 
our efforts. In the case of preschool, millions of dollars are being spent on our preschool 
program, yet its effectiveness here in WCPSS has not yet been demonstrated. When students are 
screened for next year, a comparison group should be established which can be followed over 
time through a contracted evaluation. Two options are a comparison group of those just below 
the cut (waitlisted) if the selection criteria are strictly adhered to or a group that receives a 
different pre-school model. The use of nationally normed preschool measures (such as the 
Brigance) is imperative to allow developmental comparisons over time. The evaluation 
contractor should be hired soon, before screening, to support the establishment of a solid 
research design. Data should be collected for students on the waitlist in terms of preschool 
participation; spring posttest scores on the Brigance should be obtained if possible. Student 
attendance and fall and spring testing data should be recorded consistently as part of the study. 
Measures used to monitor progress should include norms that match the program goals and 
which provide developmental norms to assess relative progress. Only one such measure was 
available for the longitudinal study, and the correct version of the pretest was not given to all 
2005-06 participants. 

Improve data collection mechanisms. In addition to establishing a quality long-term student 
design in advance, having high quality data readily available will enhance the ability to complete 
a quality study in a timely manner. Title I staff are encouraged to put a high priority on changing 
over from paper data collection to an electronic system for preschool in the coming year. 
Collecting data efficiently will make it easier for program staff to use data in an ongoing way as 
well. In the case of this study, data were not readily available and were incomplete when 
located; this cost valuable time for program and evaluation staff and slowed completion of this 
study. 
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